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PREFACE: Jaclynn Tracy 

This is a Preface to a larger collection, entitled Performance Assessment of 
Educational Leadership Programs, edited by James Berry and Ronald 
Williamson, Eastern Michigan University, and available at: 


The transformation of programs preparing school leaders is nothing short of 
startling. In response to demands for strengthened preparation, educational 
leadership programs throughout the country embarked on a courageous 
journey to align college and university courses with national standards and 
to design student performance activities that would measure whether 
students could perform as school leaders. This authors of this book draw on 
their experience at one of the nation’s largest leadership preparation 
programs, Eastern Michigan University. As often occurs in higher education 
the first steps to embrace performance activities were tenuous and 
begrudging. But as the work continued these faculty readily embraced a 
new pedagogy, became enthusiastic about the positive impact on students, 
and designed activities that both challenge students and significantly 
strengthen the instructional program. In each chapter the authors take a 
retrospective look at their experience and provide important insights into 
the improvement process. They offer a thoughtful critique of the changes in 
curriculum, instruction and assessment occurring in one program. Their 
learning offers important lessons to others embarking on this journey. I am 
proud to work with these courageous faculty and respect their commitment 
to continually improving their work with students. I am sure you will find 
this book helpful to your efforts. 


Jaclynn Tracy, Department Head Department of Leadership and 
Counseling Eastern Michigan University 


1. Performance is the New Standard in the Preparation of School Leaders; 
James Berry 

The educational administration curriculum is being recast to emphasize 
academic as well as applied knowledge. Training for the applied nature of 
educational administration has been a challenge for most graduate programs 
as they emphasized the academic and theoretical approach to the leadership 
and administration of schools. The applied nature of educational 
administration has defined the field since its inception and the teaching of 
performance behaviors has become an integral component of the 
educational administration curriculum. The educational administration 
curriculum will, in the future, reflect this aspect of teaching and learning. 


Note: This Collection has been peer-reviewed, accepted, and sanctioned by 
the National Council of Professors of Educational Administration 
(NCPEA) as a significant contribution to the scholarship and practice of 
education administration. In addition to publication in the Connexions 
Content Commons, this Collection is published in the International Journal 
of Educational Leadership Preparation, Volume 4, Number 4 (October — 
December 2009). Formatted and edited in Connexions by Theodore 
Creighton, Virginia Tech. 


Introduction 


The field of educational leadership is going through a historical shift in the 
way school leaders are trained. There is recognition that complex 
educational organizations require a deep understanding and depth of 
knowledge about how to improve learning for leadership. Expecting 
prospective leaders to provide evidence of what they know through 
performance activities plays to the strength of programs in educational 
administration. In fact, the field itself is grounded in the applied nature of 


being a practicing educator and school leader. In comparing the curriculum 
of educational administration from the last decade of the twentieth century 
to the first decade of the twenty-first the significant difference is in the 
return to the applied nature of the field reflected in performance activities 
required of students in their pre-service university training. 


During the first decade of the 21%' century programs in educational 
administration were professionally oriented (some would argue directed) 
and engaged in curricular reform through a process of accreditation that 
thrust a set of standards into the development of courses and content within 
the traditional educational administration curriculum. The Educational 
Leadership Constituent Council (ELCC, 2002) standards (originally 
developed by the Interstate School Leaders Licensure Consortium and 
introduced 1996) have been used to focus programs in educational 
administration around expectations and outcomes that was a consensus of 
best practice and knowledge about preparing pre-service school leaders. 


These standards were generally accepted as representing the broad themes 
of behavior, knowledge, and skill that were essential to leading and 
managing schools and other educational institutions. Debate raged about 
whether the standards were relevant and represented what the research and 
the literature in the field said about what educational leaders really needed 
to know and do (see for example, Hoyle, 2005; Murphy, 2005). Yet, 
regardless of one’s position in this debate, most states adopted these 
standards to frame both the curriculum of university-based preparation 
programs in educational leadership and requirements for 
certification/licensure as a school leader. 


The standards were also adopted, and occasionally adapted, by state 
departments of education as a framework for guiding state-wide 
accreditation. The ISLLC/ELCC standards have become the “coin of the 
realm” for defining expectations of educational leadership programs across 
the United States. At last count the ISLLC/ELCC standards were wholly 
adopted or served as source documents for 43 states approving educational 
leadership programs through a state approved accreditation process 
(Shipman, Queen, & Peel, 2007). The ELCC Specialty Program Area (SPA) 
standards contain student expectations that reflect academic knowledge of 


educational administration as well as performance expectations requiring 
student participation in meaningful activities that are evaluated and 
assessed. 


The standards require a set of performance activities for all students 
approved by NCATE/ELCC. These activities, however, are more than paper 
and pencil activities. Performance means a student is investigating a topic, 
demonstrating a skill, or doing an exhibition. They might require written or 
oral responses that are included in a portfolio or journal that incorporates all 
of the performance activities. 


Musical or athletic performances have been the standard for activities that 
require demanding practice at a high level of skill. The renowned Julliard 
School of Music explains the importance of student performance in seeking 
excellence: 


Performance is an essential component of the learning process at The 
Juilliard School. More than 700 drama, dance, and music events take place 
in the Juilliard building during the school year. They present individuals 
and ensembles whose teachers have nurtured, shaped, and readied them for 
the moment when they step on stage, the moment when all their training 
and preparation come into focus and the energy they give comes back to 
them in the appreciation of an audience. (Julliard School, 2008, p. X) 


Under the guidelines established for educational leadership programs 
seeking Educational leadership Constituent Council (ELCC) accreditation 
through the National Council for the Accreditation of Teacher Education 
(NCATE), performance means that a student must accomplish some kind of 
meaningful task that teaches or reflects a skill that a school administrator 
might perform in day-to-day work. 


This performance expectation does not diminish the importance of content 
knowledge required to explain the task or understand how one thinks 
through an activity. Performance, in context to meeting standards, means 
educational leadership programs must expect students to demonstrate the 


applied nature of their knowledge in order to show that they both know and 
can do the job of school administrator. 


Pre-Service University Preparation 


Aligning a curriculum with the ELCC standards has impacted university- 
based preparation programs. The renewed focus on learning-by-doing has 
emerged as central to the preparation of educational leaders. The emphasis 
on practice that was fondly referred to as war stories told by former school 
administrators who had been hired as university faculty—has now come 
full circle. The renewed focus on assuring that students not only know but 
can do has meant curricula must be developed that increasingly emphasizes 
skills and behaviors that will transfer to the applied aspects of 
administration and leadership. 


Performance assessment can be defined in several ways—as actual 
demonstrations that show learning has occurred, as results-based 
assessment, as task performances that reflect real-life situations, as the 
production of a product or performance that reflects program objectives, 
and so on. It is most useful when focusing on broad professional tasks 
rather than on a single sub-skill. Performance assessment may include 
aggregated evaluations or random samplings of candidate and/or professor 
portfolios, rubrics of projects and investigations, candidate program 
evaluations, interviews, documented observations of simulations or clinical 
experiences, peer assessments, job performance of graduates, and so on. 


Another characteristic of performance assessment is the use of higher-order 
thinking skills (e.g., analysis, synthesis, evaluation) by candidates. 
Integration of performance assessment with the instructional process allows 
assessors and candidates to interact to strengthen learning. 


Performance assessments are: 


e Essential, not arbitrary or contrived to shake out a grade. 
e Enabling, constructed to point the candidate toward more sophisticated 
use of skills or knowledge. 


e Contextualized complex intellectual challenges, not atomized tasks 
corresponding to isolated actions. 

e Representative challenges designed to emphasize depth more than 
breadth. 

e Engaging and educational. 

e Involved with broad tasks or problems that may be somewhat 
ambiguous. (NPBEA Standards, 2002, p. 12) 


The shift in preparation of educational leaders reflects earlier debate about 
the focus on such preparation. The field of educational leadership has 
always been grounded in the applied nature of leadership. Today, however, 
there is growing emphasis on learning the applied skills and behaviors of 
school administration—as it was during the early formative stage of 
development of educational administration in the twentieth century—for 
constructing knowledge about educational leadership. 


Knowledge and Performance 


In the early 1950s educational administration moved away from the applied 
nature of the field and embraced the university-based academic, scientific, 
rational, and theoretical approach to training school administrators. Griffiths 
(1959), especially, believed that the field of educational administration 
should be elevated to the status of other academic fields within the 
university. The way to do this was to turn the curriculum into a more 
academic oriented approach to training school administrators. Perhaps the 
best example of the field’s desire to achieve a higher status within the 
academy was the founding of the University Council for Educational 
Administration (UCEA) as a professional organization for those preparation 
programs emphasizing the academic, scholarly, and research orientation of 
the field. Prior to the founding of UCEA in 1956 the field of educational 
administration was represented by the National Council for Professors of 
Educational Administration (NCPEA). The very nature of UCEA’s 
existence was based upon a belief by select universities that the field must 
establish a more scholarly, research, and academic curriculum to prepare 
school administrators. 


University pre-service preparation to be a school administrator then became 
a graduate level exercise that mirrored the movement of the curriculum 
toward a more theoretical and academic approach that emphasized the 
cognitive knowledge related to the practice of school administration. 
Professors and university programs moved with the times to satisfy a 
perceived need to fit within the academy and to offer a more substantive 
academic curriculum to address the weaknesses of the prescriptive era of 
the previous fifty years (Murphy, 1992, p. 41). The thirty-four university 
programs in educational administration that became UCEA founding 
members were taking on the challenge to “improve graduate programs in 
educational administration through the stimulation and coordination of 
research, the publication and distribution of literature growing out of 
research and training activities, and the exchange of ideas” (Campbell, et al, 
1987, p. 182). 


For the past fifty years the field of educational leadership treated the 
applied nature of school administration as the handmaiden to the academic 
orientation that the field embraced during the 1950’s. Although the 
profession recognized the importance of the hands-on aspect of educational 
leadership, it also denigrated the stories, anecdotal experiences, and front 
line encounters of ex- administrators as colorful commentary and historical 
footnotes with little academic value within the higher education academy. 


Thus, aspiring educational leaders memorized theories and learned the 
cognitive and scientific approach to leadership without an intentional or 
thoughtfully constructed experience that applied learning or required 
performance to complete, enhance, and complement well-constructed 
lectures that described and theorized about actual leadership within a school 
or school district. 


An emphasis on academic learning at the expense of experiential learning 
contributed to the criticism of school leadership preparation. An invigorated 
focus on student learning and accountability for student performance led 
schools and districts to seek leaders who not only knew what to do but how 
to perform in demanding positions that required affective, behavioral, and 
hands-on skill. 


Student Performance 


After fifty years of diminished credibility of a curriculum devoted to the 
applied nature of school administration, the field has once again embraced 
performance and experience . .. within an accepted academic epistemology. 
The problem-based, experiential, constructivist, and student-centered forms 
of pedagogy that place more of the responsibility for learning on the learner 
have been embraced as important curricular components within the 
educational leadership curriculum. The field of educational leadership 
entered a period in which it recognized the applied nature of the profession 
and used a broader definition of learning to complement academic with 
applied learning. This perspective, with ties to the intellectual work of 
Dewey (1959), Lewin (1951), Piaget (1958), Rogers (1969), Vygotsky (see 
Daniels, 2001), Lave & Wenger (1991), Kolb (1984) and others emphasizes 
“the central role that experience plays in the learning area” (Kolb, 1984, p. 
20). 


The profession is now in the process of re-balancing rational, theoretical, 
and scientific with performance, activity, and experience. The problem- 
based, experiential, performance approach extends, refines, and reconnects 
the applied educational administration curriculum to the academic 
curriculum that arose in the 1950’s. The applied nature of the field is being 
reinforced by standards that have performance expectations and 
requirements. 


To imbue the educational administration curriculum with meaningful 
performance activities that replicate day-to-day experience is an attempt to 
reconnect the professional and theory based aspects of educational 
administration to its more practical and applied elements of managing and 
leading an educational organization. The adaptation of performance and 
experiential learning into the educational administration curriculum is 
returning to the very practical knowledge of what to do and how to do it and 
recognizes the importance of merging applied knowledge with the academic 
training within educational leadership preparation. 


Experience and knowledge together draw the learner toward a deeper level 
of learning and understanding that results in a personal change. As Dewey 


(1912) noted: 


Experience as trying involves change, but change is meaningless transition 
unless it is consciously connected with the return wave of consequences 
which flow from it. When an activity is continued into the undergoing of 
consequences, when the change made by action is reflected back in a 
change made in us, the mere flux is loaded with significance. We learn 
something. (Dewey, 1912, p. 163) 


Kolb (1984) viewed learning as a complex process that required 
“immediate concrete experience” . . . or the “indirect comprehension of 
symbolic representation of experience” (p. 58). Importantly, he considered 
learning, which led to new understanding as “the process whereby 
knowledge is created through the transformation of experience” (p. 38). The 
transformation of experience, he theorized, was representative of a process 
that involved a series of psychological events that required reflection and 
thought that then led to new understanding or learning. These phases of 
learning are the experiential learning cycle and include: experiencing, 
publishing, processing, generalizing, and applying as represented in Figure 
1 (adapted from Walter & Marks, 1981). 


What makes experiential learning so relevant for the field of educational 
leadership is its emphasis on practicing applied knowledge by engaging in a 
process of thinking that results in behavior change while progressing 
through an academically rigorous curriculum. The premise behind 
experiential learning is that, “Ideas are not fixed and immutable elements of 
thought but are formed and re-formed through experience” (Kolb, 1984, p. 
26). With the advent of ISLLC/ELCC standards, and the commitment to 
performance learning that is an integrated component of the educational 
administration curriculum, the field has re-established the importance of 
applied knowledge. The field has re-established its roots as an applied 
profession. 


The context for experiential learning in the educational leadership 
curriculum is to develop a deeper level of understanding by being involved 
in well crafted activities and problems that elicit behaviors and reflections 


commensurate with roles and responsibilities that are closely aligned with 
the job of school administrator within a community of practice. Lave and 
Wenger (1991) describe the importance of acquiring knowledge as an 
apprentice (guided by an involvement with a structured pattern of learning 
experiences) within this community of practice. 


A learning curriculum is situated. It is not something that can be considered 
in isolation, manipulated in arbitrary didactic terms, or analyzed apart from 
the social relations that shape legitimate peripheral participation. (p. 97) 


Figure 1: Experiential Learning Cycle 
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Adapted from: 


Walter, G. A. & Marks, S. E. (1981). Experiential learning and change. New Y ork: John 
Wiley & Sons. 


The curriculum of the educational leadership program, after fifty years of 
academic emphasis is being more closely aligned with applied learning that 
expects graduates of an educational leadership program to have a command 
of meaningful skills that can be transferred to leadership roles and 
responsibilities in education. Significantly, there is the recognized 
importance of imparting knowledge through well-crafted and meaningful 


activities that are as challenging and significant as academic work within 
the classroom. Vygotsky (see Daniels, 2001) further explained how these 
well-crafted activities could build on previous learning and scaffold into 
new learning. However, the construction of new learning was not an 
independent cognitive activity. As Vygotsky and others have written, the 
social and cultural context in which the student interacts helps to reinforce 
the more complete understanding of the performance activity within the 
educational context. 


The field of educational leadership preparation is embracing the more 
thorough education of the aspiring school administrator by challenging 
him/her to learn . . . using the both the left (more cognitive) and right side 
(more conceptual) of the brain. 


A Change in Curriculum Delivery 


Berry and Beach (2006) wrote that the field of educational leadership had 
practical, professional, and academic domains that served as a foundation 
for curriculum planning within educational administration programs (Table 
1). Educational leadership is a complex, diverse, and amorphous field that, 
ultimately, draws from many professions. The applied nature of educational 
leadership is substantively represented by a complex intersection of 
practical, professional, and academic knowledge—from dealing with an 
angry parent to making a presentation to the school board about student 
achievement—that demands a high level of performance in one’s day-to- 
day role within an educational organization. 


practical professional academic 


Educational Leadership Domains 


(Berry & Beach, 2005) 


e Each domain represents a body of knowledge within the educational 
administration curriculum. Your first item here 

e Educational administration preparation programs have struggled to 
determine the appropriate balance between each knowledge domain. 


Achilles and DuVall (1991) described the typical educational leadership 
program in the late twentieth century as focused on a limited knowledge 
base that emphasized just two areas. The first area was in the social 
sciences and included grounding in economics, geography, history, political 
science, psychology, social studies, and sociology. According to Achilles 
and Duvall, “This has led to a pseudo-scientific aura that surrounds course 
content, research methodology, and academic perspective” (p. 16). A 
knowledge base in educational administration, they suggested, should be 
framed around two distinct learning areas to have credibility and integrity. 
The social science area was basic to the profession and depended upon the 
“theoretical underpinnings that relates concepts and ideas, helps predict 
behaviors, and at least partially explains phenomena” (p. 16). The emphasis 
on obtaining knowledge within the social sciences was, as they pointed out, 
a necessary component of the educational administration curriculum. This 
area has been the primary focus of learning for the field over the last fifty 
years. 


The second area, however, requires knowledge of self and performance. 
How well does the principal, superintendent, or other educational leader 
perform in the position? This is the area that had been neglected by the field 
as it embraced the rational scientific method. “The application of concepts 
requires the processing of real time feedback. It requires continuous 
adjustment, assessment of phenomena, and alteration of courses of action. 
Performance requires the practitioner to be ‘scholar on one’s feet’” 
(Achilles & Duvall, 1991, p. 16). 


A shortcoming of the typical program in educational leadership at the end 
of the twentieth century was its focus on the pseudo-scientific rather than 
the practice of leadership. Students were expected to learn from a 
curriculum that emphasized theory over practice and measured success by 
the student’s ability to transfer knowledge through memorization of 
material from a lecture that described a prescribed practice. 


Performance Activities 


A performance activity requires an active behavior by the student. Being an 
active participant is key to the learning performance. An active performance 
similar to the one described below offers insight into how structuring 
student behavior through a meaningful activity shapes student learning. 


A Theory for Leading: Distributed Leadership 


Activity #1: Evaluating Consensus Decision-Making 


Each student will observe a district level team that is focused on an aspect 
of instructional improvement and complete a consensus protocol based 
upon team/committee performance. The consensus protocol will be used to 
observe individual leadership behavior in a group. 


Reading: Before observing the team/committee for this assignment read: 


e Spillane, J. P., Halverson, R. and Diamond, J. B. (2004). Towards a 
theory of leadership practice: a distributed perspective, Journal of 
Curriculum Studies,36(1) 3-34. 

e Berry, J. E. (2005, Winter). Professional leadership accountability: 
Evaluating the work of educational teams, Education Leadership 
Review, 6(1), 9-14. 


The student must have a prior working knowledge of consensus to complete 
this assignment. 


1. The student must have a working definition of consensus roles of 
behavior to determine the patterns of behavior within the group. 

2. The student must have an understanding of how consensus is used in 
making group decisions. 


Performance Requirements: 


1. The student will submit a written analysis that describes consensus roles 
and behaviors that were observed during the meeting. Identify strengths and 
areas of concern for this team. A tabulation of consensus behaviors will be 
aggregated and submitted in chart form indicating observed behaviors: (a) 
raw score and (b) total percentage. 


2. The student will debrief the team on its observed performance: focus on 
Task and Maintenance functions only. Include a summary of debriefing as 
part of the written analysis. 


3. The student will write a growth plan that targets improvement in 
consensus decision making for the committee/team observed. 


Activity #2: Facilitating Distributed Leadership 


Utilizing the knowledge gained from the previous observing consensus 
assignment the student will facilitate a group or team meeting utilizing the 
skills of consensus. The student will handle all aspects of organizing and 
leading a consensus group. 


Organizational and Meeting Requirements: Utilize consensus decision- 
making as the method for organizing and leading a meeting devoted to an 
aspect of education. The book Rules for Reaching Consensus is a source 
book for learning the details, guidelines, and rules of consensus: 


Saint, S. and Lawson, J. R. (1994). Rules for reaching consensus. San 
Diego: Pfeffer & Company. 


Requirements: 


1. The student will utilize another student from this class or train a member 
of the team to evaluate the student/facilitator’s consensus behavior in 
leading a team.) The student will submit a written analysis (including the 
tabulation matrix with percentage scores tabulated) that describes the 
consensus roles and behaviors observed during the meeting for the 
student/facilitator. 


2. The observer will debrief the facilitator on the facilitator’s overall 
observed performance: focus on Task and Maintenance functions only. 
Include a summary of the debriefing as part of the written analysis. 


3. The student will write a growth plan that targets improvement in the 
facilitation of consensus decision-making teams per the feedback obtained 
from the debriefing. 


Click Here to Access Task, Maintenance, and Anti-Group Functions 
Tabulation Matrix 


The educational team has become a fixture in K-12 education as an integral 
component of decision-making. In this performance assessment students 
learn the skills of decision-making in the context of the theory of distributed 
leadership. Each student will evaluate a decision making team utilizing an 
evaluative model for observing consensus behavior. The student is required 
to attend two educational meetings in a school district to assess professional 
behavior associated with the process of team decision-making. Within this 
requirement the student observes leadership behavior, analyzes it within the 
context of the culture of the organization, and reflects upon what has been 
observed. The student is then expected to perform as a facilitator of the 
observed group and critique his/her own performance as a leader. 


The knowledge, skill, and behavior required to perform this activity 
represents a dramatic shift for the systematic training of educational leaders 
away from the lecture/discussion classroom that characterized educational 
leadership training for most university programs during the last half of the 
twentieth century. It doesn’t diminish the importance of knowledge about 
distributed leadership, but extends this knowledge to an activity that 
requires an engaged student performing and applying the new learning 
about distributed leadership in a meaningful way. 


Conclusion 


Ironically, the argument emerged in the 1940’s that educational leadership 
preparation was too practitioner oriented. Too many university professors 
were hired out of K-12 administrative positions and then developed 


programs oriented around the applied nature of educational leadership. 
These professors had little trouble distinguishing the importance of an 
applied curriculum over an academic curriculum. The academic work was 
about applied practice and the day-to-day running of a school and school 
district. 


The problem with the training of educational leaders, according to 
practitioners, was that preparation shifted too far in the direction of 
academic learning during the last half of the twentieth century. The 
academic culture of the university supported and promoted this orientation 
to the point of making some programs of educational leadership preparation 
irrelevant to the practice of school administration in K-12 schools and 
school districts. 


Acquisition of performance ability has again become a recognized 
expectation to insure that what principals know and can do transfers to the 
real world of schooling. University preparation is once again connecting 
academic knowledge to the applied day-to-day responsibility of being a 
school principal or front line administrator. The match between what 
professors know and can teach is adding academic and practical value to 
what principals and superintendents need to know and do for performing 
their jobs on a day-to-day basis. 
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2. Staging the Performance: The Challenge of Authenticity in Educational 
Leadership Performance Assessment; David Anderson 

This chapter focuses on the unique challenges of performance assessment in 
the field of educational leadership. In Teacher Preparation programs, 
authentic assessment is challenging but doable. If you want to know if 
someone can teach, you should not ask them to write an essay about what 
lesson they would create given a set of materials and learning objectives. 
You should actually give them the materials, have them teach, and assess 
that performance. In Educational Leadership, true performance assessment 
is much more challenging. If we want to know if someone can lead, it is 
difficult to find a situation where she/he can lead and then have trained 
assessors evaluate her/his performance in that situation. And what products 
could be captured from that performance? This chapter addresses these 
challenges by outlining four interconnected phases for developing a 
performance assessment, involving twelve steps: Design (prioritizing the 
skill domain, choosing an assessment activity, choosing a product from that 
activity, brainstorming the characteristics of a good product, formulating 
those characteristics into a rubric, and setting benchmarks for the rubric), 
evaluation (piloting the assessment, and applying the rubric to a small 
number of students), implementation (training assessors, and applying the 
rubric to all students), and program development (evaluating the results, 
and returning to step #1). A specific example of these steps is provided, and 
the role of performance assessment in reconsidering the domain of 
knowledge/skills/dispositions for educational leadership is discussed. 


Note:This module has been peer-reviewed, accepted, and sanctioned by the 
National Council of Professors of Educational Administration (NCPEA) as 
a significant contribution to the scholarship and practice of education 
administration. In addition to publication in the Connexions Content 
Commons, this module is published in the International Journal of 
Educational Leadership Preparation, Volume 4, Number 4 (October — 
December 2009). Formatted and edited in Connexions by Theodore 
Creighton, Virginia Tech. 


Introduction 


Concerns around the quality of public education have always been a part of 
the history of the US, but they have been intensifying over the past couple 
of decades. With these concerns, the demands for accountability have 
grown: first in K-12 and now in post-secondary education (Fritschler et. al., 
2008). Accountability refers to reviewing the quality of educational 
programs and holding them to standards of quality through a series of 
sanctions and/or rewards. 


Appropriately, the quality standards that drive accountability in education 
have been grounded in student learning. So, across all the academic and 
professional fields, there has been a tremendous amount of work defining 
what students should know and be able to do. In the field of Educational 
leadership, this domain of knowledge and skills is encapsulated by the 
Educational Leadership Constituent Council (ELCC) standards (Wilmore, 
2002). 


The move to program accountability in education has shifted from a focus 
on standards (what students should know and be able to do) to a focus on 
the challenges of implementing performance activities that can be assessed 
to show student program in meeting standards. In professional training 
programs, these assessment challenges are significant. In order to 
understand why, let’s examine the challenges of assessment more closely. 


The Challenge of Assessment: Assessing Performance 


In professional fields in higher education, where practitioner training is the 
focus, the emphasis (quite appropriately) is more on the able to do than on 
the to know. Knowledge is important, but for someone training for a 
specific professional role, applying that knowledge is particularly 
important. 


Educators have a long history of assessing knowledge, but a relatively short 
history of assessing the application of knowledge. Knowledge assessment 
has traditionally been done through paper and pencil tests, preferably 
through standardized multiple choice tests if the results of these tests will be 


used for high stakes decisions, such as accountability decisions (Stiggins, 
1994). The field of psychometrics has constructed an impressive and 
sophisticated array of theories and tools to develop valid and reliable 
assessments of knowledge, including the so-called higher-order thinking 
skills (Linn et al., 1991). 


Assessing application of knowledge has been far murkier. This area is 
known as performance assessment. Performance assessment gets its name 
from the belief that, if a faculty member wants to measure how well a 
student applies core knowledge, the student needs to actually apply that 
knowledge in a performance activity, and the faculty member needs to 
capture that performance and evaluate it. 


Learning to drive can serve as an example. The goal of a good driver’s 
education class is to produce competent drivers. This involves helping 
student drivers master a body of knowledge (how to operate a car, rules of 
the road, etc.) as well as actually applying that knowledge to successfully 
drive a car. If the instructor wants to assess a student driver’s competence, 
she would give two types of assessment: a knowledge assessment (How 
well does the student driver understand the rules of the road?) and a 
performance assessment (How well can the student driver actually drive a 
car?). These two types of assessment complement each other because a 
traditional knowledge test can adequately sample across a broad domain of 
issues (which is impossible to address in an actual driving test), whereas a 
driving test can clearly indicate whether the student can actually drive. 


The challenge is that any high-stakes assessment, including performance 
assessment, must meet certain quality characteristics. These characteristics 
are often called the APPLE criteria: Administratively feasible, 
Professionally credible, Publicly acceptable, Legally defensible, and 
Economically affordable (Nyirenda, 1994). These criteria are typically 
applied to large scale high stakes assessments, but there are equivalent 
quality characteristics for classroom level assessments: 1) Purpose and 
Impact (How will the assessment be used and how will it impact instruction 
and the selection of curriculum?); 2) Validity/Authenticity (Does it measure 
what it intends to measure? Does it allow students to demonstrate both what 
they know and are able to do?); 3) Fairness (Is the assessment biased 


towards any group of students?); 4) Reliability (Does the assessment 
measure skills consistently regardless of situation and/or assessor?); 5) 
Significance (Does the assessment address content and skills that are valued 
by and reflect current thinking in the field?); and 6) Efficiency (Is the 
assessment reasonably easy and cost effective to complete and score?). 


In order to address these quality characteristics, assessment developers 
typically follow a set of steps (Herman et. al., 1992; Stiggins, 1994). There 
are four interconnected phases for developing a performance assessment, 
involving twelve steps: 


Phase 1: Design 


This initial phase involves identifying the knowledge and skills that a 
student should possess and developing a set of activities or products that 
provide an opportunity for students to demonstrate their mastery. The 
following steps are included in this phase: 


1) List and prioritize the knowledge, skills, and dispositions from the 
curriculum. (Standards) 


2) Choose an activity that will allow students to demonstrate mastery of 
each set of knowledge, skills, or dispositions. (Assessment Activity) 


3) Choose a product that captures this activity (an essay, a video, artifacts 
that identify the assessment object) 


4) Brainstorm characteristics of a good product. (Assessment Criteria) 


5) Formulate those characteristics into a rubric, including both holistic and 
analytic rubrics. (Rubric formation) Typically, assessment developers will 
brainstorm characteristics of a performance that hits the target, one that 
misses the target (since missing the target is not necessarily just the absence 
of the hit the target behaviors), and one that reflects developmental 
behaviors in between. This provides a three level rubric—a five level rubric 
can simply allow the assessor to place performances between the levels. In 


any case, any number of levels can be developed through a brainstorming 
session. 


6) Set benchmarks for assessment—what is adequate for program 
accountability purposes? (Setting Performance Standards) 


Phase 2:Evaluation 


During this phase designers use the performance assessment with students 
in order to determine whether the activity appropriately measures the 
knowledge and skills identified earlier. Two additional steps are included. 


7) Pilot the performance assessment with some students. 


8) Evaluate student work, using the rubric. Choose a set of student work 
samples that illustrate each level of each rubric. (Identifying Anchor 
performances) These anchor performances, along with the rubric, are 
invaluable tools for students to understand the targe they are expected to hit. 


Phase 3:Implementation 


The third phase focuses on implementing the performance assessment. 
Assessors are trained and the scoring rubric broadly used to assess student 
work. Two steps are included in this phase. 


9) Train assessors using the anchor performances to ensure inter-rater 
reliability; 


10) Apply the rubric to all appropriate students. 


Phase 4:Program Development 


This final phase involves examining data about the assessment to identify 
patterns that may indicate issues of fairness in implementation. It also 
includes examining again whether the performance activity appropriately 
measures the knowledge and skills identified in Phase 1. Two final steps are 
included in this phase. 


11) Examine the results—look for patterns in the results. Are there 
achievement gaps between males and females? Between minority and 
majority students? If so, there may be an issue of fairness in an assessment. 
Re-evaluate the assessment in light of any achievement gaps, and modify 
accordingly. 


12) Return to the original standard. Do the performances give any new 
insight into the appropriateness and significance of the standard and how it 
is worded? (Feedback loop between Standards and Assessments) 


A Common Misconception: An Essay is a Performance 
Assessment 


Many of the earliest forms of high-stakes performance assessments (that 
met the criteria listed above) were in the area of writing (Weigle, 2002). 
The problem with this history is that many educators continue to confuse 
written essays with performance assessments. A written essay is only a 
performance assessment product in two cases: 1) if it is an assessment of 
writing; or 2) if it is capturing an actual performance in written form. Many 
so-called performance assessments are neither. 


In the field of professional educator training, a classic approach to 
performance assessment is the infamous in-box activity. In this approach, a 
student is given a case scenario (with various hypothetical artifacts at his 
disposal and a pressing problem facing him) and asked how he would 
address this situation given the constraints and tools available. He then 
writes an essay describing a response to this situation, and must support his 
response with references to theoretical frameworks provided in the course. 
This is a fine learning activity, but it does not constitute a performance 
assessment in the truest sense. 


To illustrate why, we can return to our driver’s education example. The in- 
box activity is the equivalent of asking students to imagine that they are at a 
four-way stop where they are one of three cars that reached the intersection 
at the same time: what will they do? Although answering this question 
correctly (with reference to the appropriate driving rule) is valuable, it is not 
a substitute for actually watching a student reach a four way stop in that 
situation. Why? Because applying knowledge in a real situation is always 
more complex than such a simplified scenario allows. Even if a student 
understands the four-way stop rule, will she/he correctly process the 
information that occurs in the real situation? Will she/he correctly observe 
all the cars as they approach the intersection and focus about the timing? 
Will she/he be distracted by additional (unforeseen) factors, such as the 
pedestrian trying to cross the road, or the car riding close behind the rear 
bumper? Applying knowledge in the context of real situations is the essence 
of true performance assessment. 


Authenticity is the Key Challenge 


In teacher preparation programs, authentic assessment is challenging but 
doable (Doherty et al., 2002). If an administrator wants to know if someone 
can teach, he should not ask that person to write an essay about what lesson 
to create, given a set of materials and learning objectives. The administrator 
should actually give the teacher a set of materials, have him teach, and 
assess that performance. 


However, there are numerous practical problems with performance 
assessment in teacher preparation programs. It is often impractical for 
education professors to observe all of their students teaching in actual 
school environments. (This can be done by student-teaching supervisors 
and mentors, but not by each professor in every class). Besides, research 
has shown that direct observation (using a checklist or rubric) is often not 
the most valid and reliable method of evaluation. 


So, the solution is to capture the performance in a set of products: a 
videotape, a written analysis, and samples of student work (or other artifacts 
from the actual performance). These can then be evaluated. Research has 
shown that evaluation of a set of products often has more validity and 


reliability than direct observation (because of the ability of the assessor to 
carefully review and reflect on the products, which is impossible to do in 
real time). 


In educational leadership, true performance assessment is much more 
challenging. If leadership programs want to know if someone can lead, it is 
difficult to create and/or find a situation where she/he can lead and then 
have trained assessors evaluate her/his performance in that situation. And 
what products could be captured from that performance? 


The Quasi Performance Assessment: Staging 


In educational leadership, the challenge is to create staged situations where 
adequately appropriate performances can be attained. So, what are the 
strategies for staging? 


One approach is to create an activity that requires the student to take a 
leadership role and actually perform leadership responsibilities, even if the 
activity is not embedded directly in the core activities of the school or 
district. For example, any activity that requires consensus building could 
demonstrate leadership skills. 


Another approach is to have a student participate in an authentic activity by 
helping the actual school leader perform this activity. This would provide 
limited performance opportunities, but it would allow the student to 
experience and critically analyze actual performances in a very focused and 
intimate fashion. 


An Example: Designing a Performance Assessment in 
Educational Leadership 


Let’s return to the twelve steps and apply them to one hypothetical example 
in an educational leadership curriculum. This will be a fairly simple 
example for the purpose of illustration. 


Step 1: Although ELCC has identified the domain of knowledge and skills 
required of a practitioner, every program must prioritize these standards and 


decide where and how often they will assess each standard. In the program 
at Leadership University, the faculty put a great deal of emphasis on 
participative, democratic leadership. A key part of this is consensus 
building. So, the faculty designates two faculty members (who teach a 
course that incorporates consensus-building) to create a performance 
assessment that addresses consensus building which is considered a central 
part of a specific ELCC standard. The faculty members decide to define 
consensus-building (the knowledge, skills, and dispositions) using a 
framework that identifies the various behavior types often seen in group 
dynamics as well as techniques for bringing these types together into 
consensus. 


Step 2: The two faculty developers decide on an appropriate activity: they 
will ask each student to find a meeting in their organization where an 
important, but controversial, item will be discussed. They will then ask each 
student to approach the normal facilitator of the group, and ask if she/he 
could first observe an initial discussion around the topic, and then facilitate 
a consensus-building dialogue around this item. Next, the developers will 
ask each student to observe the beginning discussion using an observation 
tool that captures the behavior types of each participant at the meeting. 
Then, based on this observation tool, the student would (at least try to) 
apply the appropriate consensus-building techniques. Satisfied with this 
activity, the faculty developers write descriptions and directions for the 
students. Note: This is an authentic task because each student is actually 
using the consensus-building techniques in a real situation. It is somewhat 
staged because the meeting participants will realize that this is a student 
assignment, and this might affect the authenticity of their engagement in the 
process. Also, consensus building often requires numerous discussions, but 
this on-going process is not reflected in the assignment. 


Step 3: It is decided that students will submit their observation sheets, as 
well as essays documenting the techniques they used, the effects of those 
techniques, and recommendations for further consensus-building activities. 
Each essay would be reviewed by one of the other participants from each 
student’s meeting for accuracy and completeness. 


Step 4: Given the consensus-building framework, the faculty decide that the 
observation sheet should identify a range of behavior patterns across 
multiple participants in the meeting. The essay should address multiple 
techniques by clearly identifying each technique and how it was used in the 
discussion, as well as analyzing strengths and weaknesses. 


Step 5: Then, the faculty visualize an exemplary student performance and 
brainstorm descriptors of that performance, including the number of 
behaviors and techniques integrated into the performance and analysis. 
Next, they visualize a performance that misses the target for a variety of 
reasons, and they brainstorm descriptors of that performance. Finally, they 
imagine a performance in the middle and brainstorm appropriate descriptors 
at that level. In all three cases, these descriptors include adjectives that 
capture a sense of quality, not just quantity of the behaviors and techniques. 
The faculty members notice that the descriptors fall into two broad 
categories: observation and analysis of behaviors, and selection and 
analysis of techniques. They split the descriptors into these two groups to 
create two analytic rubrics. 


Step 6: The faculty decide that the middle levels of each analytic rubric 
demonstrate an adequate mastery of the skills to meet the ELCC standard. 
The top levels go beyond the expectations of the ELCC standard, and give 
room for further growth for the more advanced students completing the 
performance. They create a table that captures the three levels of both 
analytic rubrics, including all the descriptors. 


Step 7: The faculty decide on an appropriate instructional activity which 
will help students prepare for this assessment: a simulation in small groups. 
They then pilot the instructional activity and assessment in one section of 
the course. They decide that, even though the assessment may not yet meet 
the rigorous standards for external accountability (e.g., the APPLE criteria), 
the design process to this point ensures that it meets their standards for 
supporting high quality teaching and learning. In other words, it will be a 
good learning experience for the students, and the instructor of this pilot 
course will be able to grade the assessment in a fair and appropriate manner. 
Since the assessment is part of the students’ grades, the students in the pilot 
will give the assessment appropriate effort. (Note: If the assessment is used 


this way in the piloting, it should be used this way in the actual 
implementation for validity and reliability purposes.) 


Step 8: Next, the faculty developers choose samples of student work and 
assess each student individually. Then they meet and grade the samples and 
discuss/resolve any differences in assessment, until they have identified at 
least two examples of student work at each level for both rubrics. These are 
the anchor papers. 


Step 9: The two faculty members arrange a meeting for all of the faculty 
who teach this particular course (and a few other interested faculty). At this 
meeting, the two faculty developers distribute the description and directions 
for completing the activity (which was given to students) as well as the 
rubric (which was also given to the pilot students). Next, the two developers 
share half the samples of student work, and ask each faculty member to 
individually assess the samples with the rubric. Then, as a group, the faculty 
discuss their assessments until they agree on what each sample should 
receive. Then, the faculty members distribute the second half of the 
samples, to check and make sure that there is consistency among all the 
assessors. Any re-categorization of anchor papers is discussed and resolved. 


Step 10: All faculty members (both full and part-time) who teach sections 
of the course integrate the assessment into their sections. The faculty are 
given freedom in deciding how to incorporate the assessment into their 
grading practices. 


Step 11: After a year of giving the assessments, the faculty sit down and 
examine the results. They notice that students working in urban 
environments are getting consistently lower scores. In examining samples 
of student work, they realize that these students are dealing with much 
larger groups and are trying to evaluate, in depth, more individuals. The 
faculty decide to change the directions by asking students to focus on 
specific influential participants to do their analysis. This helps with the 
validity of the assessment since it reflects appropriate consensus-building 
skills. They also notice that one adjunct professor gives consistently higher 
scores. So, they sit down with this instructor and assess samples of student 
work together. During the subsequent discussion, they identify a 


misunderstanding on the part of the instructor as to the definition of one 
behavior and associated technique. 


Step 12: Having done this assessment in many sections, they convene 
another meeting of all the instructors who have given the assessment. At 
this meeting, the instructors indicate that consensus building needs to be 
tied to group communication skills, and they suggest a slightly different 
framework that incorporates both areas. With this new definition, they 
return to step 2 and make appropriate modifications. 


Clearly, these steps reflect an ongoing process. One difficulty is that as 
rubrics change, it is difficult to examine longitudinal data. However, faculty 
can balance the need for revision with the need for longitudinal analysis as 
they see fit. 


The Final Solution: A Portfolio of Assessments 


In order for comprehensive accountability, multiple types of assessments 
are needed: performance assessments, knowledge assessments, and hybrid 
assessments. A program also needs to assess a broad domain of standards 
(knowledge, skills, and dispositions), and try to assess each of these 
characteristics through multiple assessments. But how is this done? 


The best approach is to create a programmatic portfolio with course- 
embedded assessments. This requires horizontal and vertical articulation 
(scope and sequence) across the courses in the entire program. Portfolios 
are most valuable when the course embedded assessments can reveal a 
student’s progress (or lack thereof) over time. To achieve this, the 
assessments can be created in a way that reflects skill building over time 
(Koretz et al, 1993; Calfee et. al., 1996; Cummings et al., 2008). 


Yet, this is difficult in the field of educational leadership, since most 
programs cannot afford to use a cohort model where courses are taken in a 
particular sequence. Practitioners need the flexibility to take courses in any 
order, given the need to balance both personal and professional roles. 
However, well-designed performance assessments demand thick description 
and rich analysis from students. These in-depth student work samples can 


still reflect student growth if assessors add written comments that pinpoint 
individual student needs, and subsequent assessors check on these needs. 


The Future: Learning From the 12th Step—the Feedback Loop 
of Assessments to Standards 


In all assessment processes, there is a strong feedback loop between the 
standards and the assessments, as captured in the 12" step of this 
framework. Standards are written to capture a vision of what students 
should be able to do. But assessment is where the “rubber really hits the 
road”—the faculty do not really understand and appreciate how a standard 
is phrased and framed until it is operationally defined by its corresponding 
assessment. And in assessing it, they more concretely see the skills in action 
and can re-evaluate the programmatic vision. 


Although this example focuses on a very specific area, this process of 
revision should be happening more broadly in the field of educational 
leadership. The ELCC standards are thoughtfully developed and 
comprehensive, but what do they look like in practice, given the 
profession’s experience with performance assessment to date? What are 
individual programs learning in this regard? 


Overall, the field of educational leadership is just beginning its journey 
towards a standards-driven performance assessment-based curriculum. This 
will be a process of trial-and-error, but it may well lead to stronger 
programs with significantly higher credibility with the public and the rest of 
academia. 


Since assessment must be aligned with instruction, the use of performance 


assessments tends to orient instruction towards constructivist activities, 
where students are highly engaged in constructing their own knowledge. 
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3. Changing Assessment, Changing Instruction; Ella Burton 

University based school leadership preparation programs are under attack. 
Accreditation standards, federal mandates, and the standards-based 
movement are changing instruction and assessment. The changing 
landscape of leadership preparation requires higher-education programs to 
engage in complex discussions altering long-standing norms about teaching 
and learning. The challenge is to maintain a focus on students and their 
need to be highly skilled, thoughtful practitioners who are able to know and 
use the knowledge they acquire in their university degree programs. 


Note:This module has been peer-reviewed, accepted, and sanctioned by the 
National Council of Professors of Educational Administration (NCPEA) as 
a significant contribution to the scholarship and practice of education 
administration. In addition to publication in the Connexions Content 
Commons, this module is published in the International Journal of 
Educational Leadership Preparation, Volume 4, Number 4 (October — 
December 2009). Formatted and edited in Connexions by Theodore 
Creighton, Virginia Tech. 


Introduction 


University based school leadership preparation programs are under attack. 
Levine (2005) suggested that school leadership preparation programs were 
in a “race to the bottom” (p. 61). Preparation programs are also being asked 
to address more explicit accreditation standards and the Secretary of the US 
Department of Education has suggested applying standards embedded in No 
Child Left Behind (NCLB) to higher education (The Spellings Report, 
2006). This report has an ambitious set of goals as to the expectations for 
American higher education. There are five goals specific to charting the 
future of’ higher education and one of those is very specific in that 
postsecondary institutions are expected to provide high-quality instruction. 


At the heart of nearly every report suggesting change in the preparation of 
school leaders is a recommendation that programs strengthen the quality of 


teaching, gather data about student success while in the program and upon 
graduation, and adopt common performance assessments that assure that all 
students meet standards and benchmarks and have a similar educational 
experience. 


Such dramatic changes rarely come without resistance or debate. The 
standards-based movement represents a major change and as a result has 
been both vilified and praised. As institutions of higher education tackle the 
requirements of the standards-based movement there is a need to recognize 
the vast differences in the professoriate depending on whether emphasis is 
placed on teaching or engaging in research. 


Teaching and Learning in a Standards-Based Environment 


The change in the culture of assessment begs inquiry into the pedagogical 
practices that professors use in the courses they teach. To improve 
instructional effectiveness it is important to establish a framework that 
shapes how the program values instruction. An analysis of current practices 
and a review of related pedagogy support identification of appropriate 
instructional approaches. These practices must support student acquisition 
of the knowledge and skills to successfully complete the performance 
activities. Student success, including the extent they learn and grow 
intellectually, are the important considerations. 


There are prescribed steps used to design performance activities. The 
process begins with a decision about what standards should be assessed, 
determining a performance activity, establishing a scoring rubric and 
establishing a way to assess student work. Strategies for aligning course 
design and course delivery with both national and state standards, and 
incorporation of common performance assessments into courses must be 
developed (see Anderson, Chapter 2). 


Student success hinges on the alignment of instruction with assessment. 
Often this requires modification in instructional design and delivery. 
According to Marzano et al. (1992) there are dimensions of learning that 
involve the interaction of five types, or dimensions of thinking, which are 
loose metaphors for the way the mind works during learning. The five 


dimensions of learning include acquisition and integration of knowledge as 
well as the ability to use knowledge meaningfully. At the heart of the 
emphasis is performance assessment that measures student acquired 
knowledge and skill. (see Berry, Chapter 1). 


Changing Instruction, Challenging the Professoriate 


The changing role of the professoriate starts with a review of the general 
approach to teaching and learning. The challenge is to develop changes in 
response to conditions that include greater clarity about the knowledge, 
skills and dispositions desired of graduates. There is the expectation that 
teaching become research-based and faculty develop and use common 
assessments embedded within key courses in degree programs. The latest 
thinking on how people learn—differing learning styles, multiple 
intelligences—is changing the nature of teaching in higher education. 
Students have more options for when, where and how they complete 
coursework than ever before (Cross, 2005). 


Today, students attempt to connect concepts to their own life/work as they 
look for connections and relationships between text, lecture, and 
assessments. Students have an increased interest in the quality of their 
learning and are willing to critique their work on quality standards, as well 
as new knowledge, skills and dispositions developed as a result of their 
experience. 


In order to address the programmatic needs and the needs of students, 
faculty must review delivery systems, teaching methods, course materials, 
curricula, student assessment and other factors. The professoriate must 
increasingly consider a blended learning platform to accelerate learning 
with seamless access and delivery of formal, informal and self-paced 
learning resources across the continuum of learning styles. Degree 
programs that utilize blended course offerings—those that combine face-to- 
face instruction with online learning and reduced classroom contact hours 
have the potential to increase student learning. 


The instruction that faculties provide, the intellectual climate that they 
create, and the policy decisions that they make, should all start with the 


question, “But will it improve students’ learning?” (Cross, 2005). What is 
clear is that students who are actively engaged in learning for deeper 
understanding through performance activities are likely to learn more— 
because of this engagement—than students who are not. The evidence is 
clear that students have to find their own answers by working through their 
own pathways of knowledge. 


Emphasis on student learning requires that faculty know what to look for 
(through research), to see themselves as life-long learners (self-reflection), 
and to be much more sensitive to the range of learning styles present in 
their students (Cross, 2005). 


The task facing the professoriate is to move towards accountability by 
constructing course syllabi with an emphasis around national and state 
standards. In school leadership that means courses must address the 
standards of the Leadership Policy Standards (CCSSO, 2008) and 
Educational Leadership Constituent Council (ELCC, 2002). Common 
assessments that reflect the knowledge, skill and understanding of the 
students who have completed identified courses must be developed. Data 
about student performance on the assessments must be collected and used to 
guide program improvement. These data can prove informative when 
evaluating the quality of the teaching and learning experiences of students. 
Accreditation requirements mean development of assessments, rubrics 
(scoring guides) and a method of gathering that data that will show 
evidence those students know their area of study. 


As degree programs implement standards and embed performance 
assessments in the course offerings, faculty face the need to examine their 
own instructional skills and give consideration to research-based strategies 
for increasing student achievement. Through the lens of a researcher, 
teaching must be acknowledged as both a skill and an art. Marzano , 
Pickering and Pollock (2001) identified nine instructional strategies linked 
to improved student learning. In Classroom Instruction that Works: 
Research-Based Strategies For Increased Student Achievement the authors 
provide examples of how to utilize these strategies when considering best 
teaching practices. Questions about successful teaching led to inquires high 
quality classroom learning. What works in education? How do we know? 


How can teachers find out? How can education research find its way into 
the classroom? How can we apply research to help the individual student? 
arise in most programs and instructors do not have time to find the answers. 
Marzano, Pickering, Pollock (2001) have examined decades of research 
findings to distill the results into nine broad teaching strategies that have 
positive effects on student learning. These strategies are: 


e Identifying similarities and differences 

e Summarizing and note taking 

e Reinforcing effort and providing recognition 
e Homework and practice 

e Nonlinguistic representations 

¢ Cooperative learning 

e Generating and testing hypotheses 

¢ Questions, cues, and advance organizers 


These strategies are appropriate for all age levels and have positive effects 
on student learning. None of the identified strategies are new; however 
Marzano’s research provided evidence through a meta-analysis that 
emphasized how students gain knowledge when these instructional 
approaches are embedded into the teaching and learning experience. To 
assure quality instruction and to support expectations for greater 
accountability, faculty must embed the best research practices into course 
preparation and delivery. Such an approach affirms the learning experience 
and includes the knowledge, skills and dispositions that cause students to 
synthesize learning in a way that adds value to both their careers and 
personal lives. 


Developing the capacity to teach more effectively, to present information 
and gather data about student learning is becoming the norm in higher 
education. It is essential that faculty know how and when to best apply 
effective strategies in the instructional program and to understand that these 
strategies can be used both in designing original lessons and as 
interventions when other strategies are less successful. 


Understand the Adult Learner 


Knowledge about the adult learner is essential. “An understanding of the 
principles of adult learning is part of being effective” (Lieb, 1991). The 
knowledge and skill students are expected to learn must be delivered within 
the frame of adult learning. Professors need to recognize that elements of 
respect and relevancy provide an environment where adults learn best 
(Lieb,1991). There is little chance that academic work will become a part of 
the lifelong experience for the adult if the “special needs and requirements” 
(Lieb, 1991) are not part of the teaching and learning process. According to 
Lieb, there are four critical elements of learning—motivation, 
reinforcement, retention and transference—that must be addressed to 
maximize adult learning. Effective instructional practice grounded in 
knowledge of adult learners often leads to comprehension of a few general 
principles with a thorough grounding in the application and process to a 
variety of concrete details. Such an approach helps students to remember 
how to apply the key principles to immediate circumstances. It is as 
important to acknowledge that adult learning theory and principles are very 
operative in higher education. Zemke & Zemke (1984) wrote that the 
knowledge about adult learning might be divided into three basic divisions: 
(1) things we know about adult learners and their motivation, (2) things we 
know about designing a curriculum for adults, (3) and things we know 
about working with adults in the classroom. 


Identifying success competencies needed for professional leadership is also 
critical. An established a set of competencies that identifies the skills of 
successful leaders in an educational setting is reflected in the work of 
Eichinger et al. ( 2003). The Education Competency Wheel is a graphic 
representation of six factors that, when incorporated into the adult 
leadership skill set, create a greater opportunity for success in one’s 
professional career and for becoming an integral member of a work team. 
Completion of a degree in educational leadership from an accredited 
university will prepare the student in competencies anchored in the six 
factors of courage, results, operating skills, strategic skills, individual 
excellence, and organizational skills. 


Blended Learning 


Blended Learning is another format that can be used to strengthen 
instructional effectiveness. The Virtual Professor (Dec, 2004) reports that 
faculty have experienced positive effects in their classroom instruction 
when using online teaching strategies. Faculty use technology in numerous 
ways: Power Point presentations, course web pages, web-based courses and 
integrating the use of computers into the classroom. The Foundation 
Coalition, one of eight engineering coalitions funded by the National 
Science Foundation—an organization committed to improving curricular 
and instructional effectiveness in higher education—suggests placing the 
technologies in the hands of students on a routine basis to facilitate 
learning, change learning outcomes, and reshape the learning process (NSF 
Coalition, 1993). Reshaping the learning process utilizing non-traditional 
applications for learning requires large investments of time for developing 
online learning versions of courses. It requires re-conceptualization of the 
content and learning objectives. 


Creating performance assessments helps students demonstrate their learning 
and supports the application of that learning to “real-life” situations. 
Additionally, asking students to reflect on and think about their learning can 
bring meaning to the content of courses. The following prompts provide 
examples of ways to promote student reflection. 


Knowledge and Understanding: What new knowledge or deeper 
understandings did you acquire as a result of participating in this 
class? What can you present as evidence of your learning? 


Attitudes and Dispositions: How were your attitudes and 
dispositions affected by your participation in this class? Where did 
you undergo the greatest change in orientation or thought? What 
attitudes and dispositions were affirmed? 


Personal Insights: What was the most valuable insight you learned 


about yourself as an educational leader through your participation in 
this class? 


Application: What will you do to apply what you leammed in this 
class? 


Effort: How much effort did you put into this class and to what 
degree are you satisfied with your learning? 


Promps for Student Reflection 


Framing a Learning Activity 


The changing landscape of leadership preparation requires higher-education 
programs to engage in complex and often contentious conversations. For 
many problems this means altering long-standing norms that ignored 
divisions and avoided substantive discussions of challenging issues. 


An expectation for accountability is pointing students to apply their 
learning in “real-life” settings. 


Step 1 — Define learning 


One of the most critical initial steps at repositioning the teaching and 
learning process is to agree upon a definition of learning. Although there 
are many ways of defining learning, the focus is on the person doing the 
learning. This is a significant shift to a focus on student outcomes (learning) 
rather than program inputs (faculty, texts, courses). Boyd and others suggest 
that learning is the act or process by which behavioral change, knowledge, 
skills and attitudes are acquired (1980, pp. 100-101). 


Step 2 — Develop a conceptual framework 


Clarity about mission, beliefs, purpose and goals, the knowledge base, 
performance expectations and performance assessments are essential 
preconditions to the work on syllabi, curriculum, teaching and learning and 
assessment. Conceptualizing a researched-based way for faculty to address 
teaching and learning creates a framework for engaging in action research, 
analyzing and understanding data about teaching and learning and creation 
of a data —driven system that improves as a result of the data collected. 


In an attempt to assure success in all areas, it is necessary to explore ways 
that university professors continue to strengthen and enhance their teaching 
effectiveness. Involving faculty in continuous review processes necessary 
for capstone courses in the concentration area of the program is one 
solution. In the updating process, attention is given to real life experiences 
as a part of the activities and learning experience.Faculty are using higher 
levels of technology to push the envelope on learning through pod casts, 
video streaming, online chats, and common assessments following course 
completion. Putting systems in place for assessing and measuring student 
learning is the best indicator of the commitment to accountability. The 
development of a set of common assessments will serve as the catalyst for 
the professoriate to change the way students experience teaching and 
learning. Instruction must address the challenge of authentic assessment. 
Instruction must match expected outcomes and be brought together through 
a variety of instructional strategies that are intentional, purposeful and 
deliberate. This is an invitation to abandon what we think we know about 
teaching and learning and develop a vision for where we are going with 
program needs as they relate to student learning and acquiring leadership 
knowledge, skill and disposition. 


Step 3 — Work on the Syllabi 


Once faculty develop a shared definition of student learning and a common 
conceptual framework for the program, the work to redesign student 
learning experiences begins. Most often it begins with the course syllabus. 
The syllabus communicates how student learning develops and progresses 
during the course. It also provides clarity about expectations and how 
student learning will be assessed. The master syllabus may require 


continuous revision as best practice changes. Therefore, the syllabus must 
be explicit about performance activities, the ways they are assessed, and 
any opportunities for students to revise and resubmit their work. 


Step 4 — Align Instructional Strategies 


Once a course is constructed there is a need to align instructional strategies 
and techniques with course goals and embedded performance assessments. 
Diagnosis of current instructional approaches may be needed. One effective 
strategy is to set aside time for an instructional diagnosis and to invite 
students to participate in these sessions. Providing time for conversation 
among faculty can build connections that strengthen and enhance teaching. 
Providing time to attend local, regional and national conferences, engaging 
in research, and utilizing university faculty development centers are 
avenues that support excellence in teaching. Promoting better thinking 
using higher order thinking skills is central to the role of higher education. 
Thus increased awareness that problem-solving skills must be an integral 
part of the course delivery is important. 


Final Thoughts 


The challenge to strengthen the preparation of school leaders is central to 
the school improvement conversation. As a result, school leader preparation 
programs are faced with increasing demands and mounting pressure to alter 
the content and structure of their programs. 


As conversations proceed they invariably result in confronting long- 
standing norms about the role and function of higher education, about the 
freedom of individual faculty to design courses, and about collection and 
use of data that may reveal trends and patterns among courses and faculty 
that were never discussed. 


The challenge is to maintain a focus on students and their need to be highly 
skilled, thoughtful practitioners able to both know and use the knowledge 
they acquire in their programs. 
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4. Designing Performance Assessments: A Guide for Faculty; Gary Marx 

This chapter outlines a strategy developed by the Educational Leadership faculty at 
Eastern Michigan University (EMU) for designing performance assessments based on 
standards for Educational Leadership. The strategy is not intended to suggest a model of 
best practice. Rather, it is presented as one example of how this work might be done and 
was written to assist faculty at other institutions confronted with the challenges of 
developing performance assessments for Educational Leadership curriculum. 


Note:This module has been peer-reviewed, accepted, and sanctioned by the National 
Council of Professors of Educational Administration (NCPEA) as a significant 
contribution to the scholarship and practice of education administration. In addition to 
publication in the Connexions Content Commons, this module is published in the 
International Journal of Educational Leadership Preparation, Volume 4, Number 4 
(October — December 2009). Formatted and edited in Connexions by Theodore 
Creighton, Virginia Tech. 


Introduction 


Standards and curriculum are related but they are not the same. The written curriculum 
for a program describes what is most important to teach and provides specific guidance 
about the scope and sequence of instruction (Squires, 2005). The basic structure for 
curriculum in school administrative preparation programs typically consists of a required 
series of courses organized around a conceptual framework or a set of functional 
responsibilities associated with the role of principal or superintendent. The theoretical 
frame or functions guide the distribution of content across the required courses to ensure 
that students encounter what they need to know and be able to do as they progress 
through their program of study. The original organizing principles for a program 
curriculum may be tacit or explicit and determine the scope and sequence of instruction. 


Standards specify desired program outcomes but they do not indicate how the standards 
should be met; the means to outcome achievement is the realm of curriculum. Standards- 
based reform can be thought of as an attempt to standardize the structural framework used 
to organize the curriculum, thus influencing the quality of instruction and ultimately 
improving the quality of practicing school administrators. The goal of standards-based 
assessment at the program level is to collect data about the performance of participants in 
the program in relation to the expectations, analyze the results, and use that information 
for purposes of program development. 


University program accreditation agencies such as the National Council for the 
Accreditation of Teacher Education (NCATE) and many state program approval processes 
currently utilize a standards-based methodology. This chapter provides guidance for 


faculty who must respond to the mandated use of performance assessments by 
accreditation and state program approval agents to document that graduates from their 
Educational Leadership programs have met some set of standards. 


The examples in this chapter use the performance expectations and indicators for 
educational leaders recently published by the Council of Chief State School Officers 
(CCSSO, 2008) as a referent set of standards to illustrate the process of performance 
assessment design at the leadership program level. This set was selected because the 
Educational Leadership Constituent Council (ELCC), the Specialized Professional 
Association for Educational Leadership programs for NCATE, will use CCSSO 2008 
performance expectations as a foundation for its seven-year review of program standards 
and revised ELCC standards are anticipated in 2009. The steps in the assessment design 
process will be the same regardless of which set of standards or performance expectations 
faculty are required to use. 


The chapter provides specific guidance for completing the six steps in the design phase of 
quality performance assessment described by Dr. Anderson in Chapter 2. It begins with a 
description of how to use a two-dimensional matrix to help determine where and how 
often to assess each standard and provides suggestions for how to address overlapping 
standards and unequal distribution patterns. The next section presents an example of a 
performance assessment to illustrate how one develops an assessment activity and 
identifies the product of assessment. Guidelines for how to develop assessment criteria, 
write a scoring rubric, and establish evaluation performance standards are then provided. 
The chapter concludes with a brief overview of the evaluation, implementation and 
program development phases of performance assessment as they relate to design. 


High Quality Performance Assessment: The Design Phase 


As discussed in Chapter 2 there are multiple steps in designing a performance activity. 
The following sections describe the six steps and utilize an example of a performance 
activity developed at Eastern Michigan University to illustrate the process. 


Step 1: Decide Where the Standards Should be Assessed 


The first step in the design process is to determine where and how often priority standards 
will be assessed. It is helpful to start with an examination of how well the curriculum in 
place is currently aligned with the expected outcomes in the standards document and then 
make adjustments where necessary. A matrix is a good tool for critically analyzing how 
particular outcomes are currently distributed across courses in the program and 
identifying gaps that may exist. 


Table 1 uses Performance Expectation 2 from CCSSO 2008 to illustrate the matrix 
organization. Regardless of the standards document used, it is important that the items 
included in the matrix utilize the finest grain descriptions available. Each CCSSO 2008 
performance expectation is subdivided into elements and each element includes a set of 
specific indicators of performance. Since the indicators for the various elements represent 
the smallest units in the set, brief descriptions of each indicator along with a reference 
code are listed down the left side of the matrix in the example. The various courses 
included in the program curriculum are then listed in separate columns across the top of 
the matrix. In this example, Instructional Leadership and School Improvement are 
hypothetical classes that deal with topics related to the supervision of instruction and 
school reform respectively. In actual practice, the matrix would include all of the standard 
elements and all of the courses required by the program of study. 


Performance 
Expectation 
2: Teaching 
and 
Learning 

Curriculum Offerings 
Performance 
Expectations 
and 
Indicators 
for 
Education 
Leaders 
(CCSSO 
2008) 


Element A. Strong Professional Instructional School 


Culture Leadership Improvement a 


Indicators 


Commitment 
to high 
expectations 


Guides 
standards- 
based 
professional 
development 


Models 
collaboration 
that improves 
practices 


Develops 
resources to 
build 
professional 
culture 


Supports staff 
examination 
of beliefs & 
practices 


Provides 
ongoing 
feedback 


Guides 
individual 
professional 
development 
plans 


Element B. Rigorous Curriculum and 


Instruction 


Indicators 


B-1 Develops Xx 
shared 
understanding 
of rigor 


Guides 
B-2 alignment Xx 
process 


Monitors 
curriculum & 
instructional 
system effects 


Uses research 
& data-based 
strategies & 
practices 


Element C. Assessment and 
Accountability 


Indicators 


Uses data to 
improve the 
C-1 quality of Xx 
teaching & 
learning 


Uses varied 
sources & 
kinds of 
information to 
guide 
evaluation 


C-3 Guides 
regular 
analyses & 
disaggregation 
of data 


Uses systems 
to monitor & 
C-4 analyze X 
assessment 
results 


Interprets data 
and 
communicates 
progress 


Sample Distribution and Alignment Matrix 


Next, faculty review course syllabi and indicate which of the elements in the matrix are 
intended outcomes of instruction in the various courses by placing an “X” in the 
appropriate cell. The completed matrix is then analyzed to determine patterns of 
distribution and identify any gaps where certain elements are not being addressed. In the 
example, there are no gaps for Performance Expectation 2 and only one indicator is taught 
in both of the two courses listed. The patterns in the sample matrix also show that 
assessment and accountability in this hypothetical program curriculum are topics 
addressed apart from curriculum, instruction, and professional development. This 
mapping process graphically illustrates the distribution of performance indicators across 
courses and the matrix is a good way to provide accreditation or program reviewers with 
evidence that the curriculum is aligned with the standards once any required adjustments 
have been made in response to the initial findings. 


During the construction of the alignment and distribution matrix it is likely that some 
indicators will be listed as intended outcomes in multiple courses. These overlaps occur 
because standards documents are organized around performance categories and the 
curriculum for many leadership preparation programs are organized around traditional 
course offerings, each of which can cover a range of performances. Where performance 
indicators overlap multiple courses, faculty are faced with a decision regarding whether to 
use multiple assessments of the same performance or assign each indicator to a unique 
course and assess the performance only once in the program. 


When making a decision regarding the final distribution of indicators, faculty should 
consider the total assessment burden associated with each course. In programs organized 
around traditional course offerings, the assessment burden will usually be greater in 
classes that emphasize the application of knowledge and skills than in classes that focus 
on the development of theoretical frameworks. This bias toward relevance and 
practicality is embedded in the standards documents themselves, which emphasize the 
enactment of leadership. 


To illustrate assessment burden, consider a curriculum where courses are organized 
around the standards categories themselves, such as offering a course called Teaching and 
Learning that addresses the indicators included in performance expectation 2 as shown 
above. In this case the assessment burden will be quite large because 16 total indicators 
must be assessed. When courses in a curriculum are organized by content or functional 
role, offering one class on instructional leadership and another on school improvement as 
shown in the example above, the assessment burden is likely to be less but still unequal 
within courses (12 indicators versus 5 indicators that must be assessed in this example). 
Although multiple measures enhance assessment validity, faculty should consider the 
practical aspects of how much time will be required to assess student work, and based on 
our experience, the time commitment is significant where multiple indicators are 
involved. 


One way to reduce the assessment burden is to redistribute performance indicators or 
defining certain indicators as secondary outcomes in some of the courses. In the example 
above, using data to improve the quality of teaching and learning might be assessed as a 
primary outcome in the School Improvement course based on a rationale that the content 
emphasizes data and information management. The use of data to inform practice would 
still remain a content topic in the instructional leadership class but it would not be 
included in the performance assessment for that class. Once these decisions are made, the 
matrix should be updated to clearly communicate how overlaps have been addressed. 
Removing the “X” in the matrix row or coding the indicators that are designated as 
secondary outcomes would be ways to communicate where each of the indicators will be 
assessed. 


This section briefly outlined a procedural method and provided suggestions for aligning 
and distributing standards within the leadership preparation program curriculum. The 
procedure ensures that students have the opportunity to acquire the leadership knowledge 
and skills associated with a standards document as they complete their academic work. 
Deciding how each performance indicator will actually be assessed is the focus of steps 2 
and 3 in the design process. 


Step 2: Determine the Assessment Activity 


The primary design challenge for faculty in educational leadership programs involves 
creating opportunities for authentic performances that allow students who do not serve in 
formal leadership roles to demonstrate their competence as leaders. The first task is to 
identify the key performances that are associated with each expectation and the place to 
start is with a close examination of the standards document and any available supporting 
material. 


CCSSO 2008 provides a narrative description of each performance expectation statement 
along with a list of elements and performance indicators intended to link the generalized 


description of the expectation to the practice of educational leadership. Table 1 above 
listed the three elements associated with the teaching and learning expectation: (a) strong 
professional culture; (b) rigorous curriculum and instruction; and, (c) assessment and 
accountability. 


By reviewing the full text of the indicator descriptions listed under each CCSSO 2008 
performance expectation anyone familiar with common practices in the field should be 
able to identify things educational leaders do when they are engaged in these aspects of 
their work. Simulating the challenges of authentic leadership practice so that students can 
demonstrate competence is the goal of good performance assessment design. The task is 
to create a context where students are required to demonstrate the identified leadership 
activities and behaviors. Once an assessment activity has been identified, the next step is 
to decide how the performance will be documented so an evaluator can make judgments 
about quality. 


Step 3: Identify the Assessment Object 


By definition, performances involve observable behavior so the assessment activity must 
require students to act in ways that demonstrate competence in meeting the associated 
standard elements. Ideally, these performances will occur in real school settings so the 
assignment must also include some mechanism for presenting tangible evidence of the 
performance that can be reviewed and evaluated by faculty. This can include a wide range 
of objects or deliverables including such things as journals, videos, portfolios, plans, 
presentations, podcasts, and so on. 


The Field Based Teaching and Learning Project in Table 2 illustrates a performance 
activity written to address the elements that were assigned to the hypothetical 
Instructional Leadership course in the alignment matrix shown in Table 1. Thus, the 
assessment tasks address all of the indicators for professional culture and rigorous 
curriculum and instruction (elements “a” and “b”), and the “data use” element from 
assessment and accountability (element “c”). The narrative for the CCSSO 2008 Teaching 
and Learning Performance Expectation and the complete list of elements and performance 
indicators can be viewed in the Appendix. 


The Field Based Teaching and Learning Project assessment activity is composed of five 
interrelated assignments in the areas of teaching and learning that require graduate 
students to apply leadership skills that enable and encourage reflective professional 
practice on the part of teachers in a school. Each assignment in the example begins with a 
description of the required performance and concludes with the evidence (object) that 
must be provided for final evaluation. The elements and indicators from the Teaching and 
Learning Performance Expectation (CCSSO 2008) that are assessed by the assignment are 
referenced within the parentheses at the end of each assignment. Notice that the 
description of the assessment activity uses words taken directly from the CCSSO 2008 


document whenever possible, which makes alignment between assessments and standards 
transparent. 


The assignments in the Teaching and Learning Project example use professional 
collaboration among peers as the context for eliciting the various behaviors. Similar 
assignments could be designed using the context of a clinical cycle involving a 
preconference, observation and post conference if the intent of the project was to simulate 
supervision of instruction. The idea is to find some meaningful and practical way that 
students can apply their knowledge and skills. In addition to being relevant, the tasks 
should also be motivating, challenging, and achievable. 


When the decision about how to assess a performance indicator is finalized, the focus of 
design shifts to the evaluation process itself. Steps 4 through 6 illustrate how to develop 
an assessment rubric, a set of scoring guidelines intended to inform students regarding 
expectations and serve as the evaluative lens. 


Sample Performance Activity/Field Based Teaching and Learning Project 


1. Recruit 1-2 peers in the workplace to join you in a learning community and work as 
a team to complete the assignments in this project. Maintain a reflective journal of 
your activities that includes a brief description of actions taken, reflections on the 
leadership behaviors you exhibited, and thoughts about how your leadership actions 
might be refined or strengthened. (A-3, A-4) 

2. Using standards-based accountability data, identify a problem of instructional 
practice at the classroom level that is related to a broader issue of student learning 
and performance within a particular content area in the school. Submit a written 
problem statement that includes a description of the procedures and processes used 
by the team to identify this as an area of concern and how the problem of practice is 
situated in the broader school improvement plan. (C-1, A-1) 

3. Select a unit of study from the district curriculum for the content area you identified 
above and conduct two audits: 1) to determine alignment of curriculum, pedagogy, 
and assessment; and, 2) to determine alignment of the unit outcomes with state 
content standards and benchmarks. Prepare a written report of your findings and 
make recommendations for enhancing the unit through the use of technology and 
research based instructional strategies in ways that fully accommodate learners’ 
diverse needs. (B-1, B-2, B-4) 

4. Collaborate with your team to address the problem of practice you identified in your 
own work setting using observation and feedback strategies and practices to analyze 
student work, monitor progress, and redesign curriculum and instruction as needed to 
meet diverse needs. (A-6, A-3, B-1, B-3, B-4) 

5. Reflect on your experiences completing this project and evaluate your current level 
of knowledge and skills as instructional leaders. Use the results of the evaluation to 
guide development of a professional growth plan for your team that reflects an on- 
going commitment to continuous improvement of teaching and learning. Prepare a 


written summary of the team discussion and an action plan that outlines specific 
goals and agreed upon expectations for how you will support each other’s learning 
over time. (A-2, A-5, A-7) 


Step 4: Define the Assessment Criteria 


Rubric design begins with a determination of the criteria that will be used to make 
judgments about quality. Criteria specify expectations regarding the types of behavior or 
attributes that should be evident in the performance. For example, prior to the 2006 
Winter Olympics figure skating free skate performances were rated based on two criteria: 
technical merit and presentation. 


When writing performance assessments to professional standards documents, the 
standards themselves specify the major assessment criteria. In the curriculum and 
instruction element from CCSSO 2008 reprinted in Table 3, the products of performance 
must include evidence the student developed shared understanding, ensured alignment, 
monitored effects of curriculum and instruction, and used data-based strategies and 
research in the process. 


CCSSO 2008: Expectation 2, Element B: Rigorous Curriculum and Instruction 


B-1 Develops shared understanding of rigor 

B-2 Guides alignment process 

B-3 Monitors curriculum & instructional system effects 

B-4 Uses research & data-based strategies & practices 
CCSSO Expectations 


When criteria are explicit and clear, inter-rater reliability is likely to be higher. As 
discussed in Chapter 3, clear criteria also inform instruction and guide student learning. 
However, clarity is usually associated with specificity and writing detailed criteria is labor 
intensive. For example, consider what would be involved to establish specific criteria for 
evaluating the degree to which a student developed “a shared understanding of rigorous 
curriculum and instruction” in a reflective journal. Most departments will likely defer 
developing criteria statements and in many cases the standards descriptions will become 


the de facto set (at least initially). The question that must be addressed by each 
department is how specific the criteria must be to guide evaluation and contribute to 
program development. Criteria can be refined as departments study performance results 
and examine student work in later steps in the process. When criteria have been selected, 
the next task is to incorporate the criteria as statements in an assessment rubric. 


Step 5: Develop the Assessment Rubric 


An assessment rubric identifies the characteristics of possible responses along a 
continuum from exceptional to unacceptable. The rubric can be designed to address each 
assessment criterion individually or combine a number of criteria for a holistic evaluation. 
Decisions about individual versus holistic scoring will affect the complexity of the 
evaluation task. Although a case can be made for individual criterion scoring, our 
experience disclosed that the behaviors associated with the assessment products we 
received from students were complex and variable so it made more sense to evaluate how 
well multiple expectations were integrated in a student’s response using a holistic 
approach. A related decision in rubric design involves how many performance levels you 
will use for each criterion and what you will name them. Too many categories can make it 
difficult to describe how one differentiates performance so we recommend starting with 
three: Exceeds Standards, Meets Standards, and Does Not Meet Standards. 


One of the acknowledged purposes for developing performance assessments is to provide 
evidence to accreditation agencies that students have met the criteria for professional 
licensure or certification. Among other things, accreditation reviewers will be looking for 
alignment between the standards expectations and the performance assessments used to 
judge competence. Incorporating the language of the standards in the rubrics used to 
evaluate student performance makes the logic of alignment transparent and simplifies 
accreditation review. However, using the language of the standards often results in 
generalized descriptions of the evaluation criteria, which is not particularly helpful for 
students trying to write to the rubric. Providing students with templates and sample 
responses for each assignment are ways to overcome vague rubrics written to ensure 
alignment is clear. 


Begin writing rubrics with a descriptive statement of the knowledge or skill being 
assessed (as specified in the assessment criteria) and then adding words and phrases that 
depict the degree of performance involved at that particular level. In the following 
example of a rubric written for assignment 1 in the Teaching and Learning Project we 
decided to incorporate the standards language to ensure a reviewer can Clearly identify the 
elements and indicators that are being assessed. We start by using the basic language of 
the standard as the statement for Meets Standards and then downgrade and embellish the 
descriptions for Does Not Meet and Exceeds. 


Sample Rubric for Teaching and Learning Project Assignment 1 


For the first assignment students had to recruit 1-2 peers to join them in forming a 
learning community that worked as a team to complete all of the assignments in the 
project. The assessment product was a reflective journal that included brief descriptions 
of actions taken, reflections on the leadership behaviors exhibited while forming and 
facilitating the team, and thoughts about how the student’s leadership actions might be 
refined or strengthened. 


The following two indicators from the professional culture element in the Teaching and 
Learning Performance Expectation (CCSSO 2008) were assessed using this performance: 


Assignment 1: Reflective Journal Summary Report Template 


e “Models openness to change and collaboration that improves practices and student 
outcomes” (A-3), and 

e “Develops time and resources to build a professional culture of openness and 
collaboration, engaging teachers in sharing information, analyzing outcomes, and 
planning improvement” (A-4). 


Here is a rubric sample statement for “Meets Standards”: 


Works with at least one peer in the workplace to complete the assignments in the project. 
The reflective journal summary report demonstrates openness to change and the ability to 
build a professional culture of collaboration that improves practices and student outcomes 
by engaging teachers in sharing information, analyzing results, and planning 
improvement. 


Note how the statement uses the language of the standard element indicators to establish 
the descriptive criteria for acceptable performance. Multiple indicators assessed by the 
same assignment are often related in some way (in this example, collaboration is the 
organizing theme) so the rubric writer can either integrate the related indicators within 
one sentence as we did here or write separate sentences for each indicator. 


Successfully recruiting and engaging at least one other person in the workplace in 
activities intended to improve teaching and learning is evidence that the student can 
actually lead this work and thus, meets the standard. Given that most students would have 
to accomplish this absent formal authority is challenging for novices so the rubric for 
unacceptable performance (Does Not Meet Standards) was written to make it possible for 
people to meet the standard at varying levels. The content of the reflective journal is 
intended to provide details regarding the leadership processes and practices employed by 
the student so it should be possible to make a judgment about the existence of a 
leadership strategy even if participation results are sparse. 


A rubric statement example for “Does Not Meet Standards”: 


Is unable to recruit anyone to collaborate on completing even some part of an assignment 
in the project. The activities cited in the reflective journal summary report lack any clear 
leadership strategy for building a professional culture of collaboration that improves 
practices and student outcomes. 


The final step is to determine criteria for exceptional performance (Exceeds Standards). In 
our example, the emphasis was placed on the results of the leadership strategy, i.e. greater 
involvement on the part of people in the workplace and actual experimentation and 
change on the part of participants. 


A rubric statement example for “Exceeds Standards”: 


Works with two or more peers in the workplace to complete the assignments in the 
project. The reflective journal summary report documents the development of a 
professional culture of collaboration and includes examples of changes in practice that 
resulted from teachers sharing information, analyzing student outcomes, and planning for 
improvement. 


The use of a common template to organize student responses is strongly recommended. A 
template not only gives students an outline for presenting their work that aligns with the 
assessment rubric, it also makes evaluating the work easier for faculty. The template 
structure standardizes how students respond to the assignment and lets the assessor know 
where to look for evidence that key components were mastered. 


The following example is a template intended to structure the reflective journal that 
faculty must review to evaluate student performance using the sample rubric developed 
above. 


Assignment 1: Reflective Journal Summary Report Template 


1. Introduction and background information: A brief overview of your workplace and 
pertinent facts related to your efforts to recruit peers to participate in the project. 
(Basic school demographics, your role, the peer group you targeted, time spent, etc.) 

2. Description of the leadership strategy you used to engage others in improving 
professional practice to increase student learning, and your rationale. 

3. Narrative description of what happened. Briefly specify who was involved, the 
nature of the problem of practice you identified for this project, and what you were 
able to accomplish. 

4. Analysis and interpretation of what happened. 


5. Summary of overall meaning. (What is the significance of this experience? What did 
you learn?) 


Step 6: Define the Assessment Standards 


This step addresses the program accountability question, “What level of performance is 
good enough?” The performance assessment and scoring rubric establishes a common 
frame of reference for professional competence but faculty must determine the knowledge 
and skill level required for mastery; i.e. when the performance meets expectations. To use 
Dr. Anderson’s example from Chapter X, this is similar to the decision that a driver’s 
training instructor must make when assessing a student’s readiness to acquire a license. 
The standards are transparent and well known but it is up to the instructor to judge when 
the student knows enough and possess at least the minimal skill level to ensure they will 
not be a danger to others on the road. In this context, “minimal” is a passing grade, 
essentially indicating that the student met the standard at an acceptable level and is 
deemed competent to engage in the practice. 


One way to begin this process is to submit the draft performance assessment documents 
for departmental peer review and discussion. This approach is a good way to get an initial 
sense of perceived face validity and how well the design elements align with the 
standards. During these discussions it would be helpful to engage in scenario building to 
explore what the baseline performance might look like. After field testing and 
implementation (Steps 7 — 10), continue checking the quality of student work, particularly 
marginal performances, to test the viability of the “cut point” that is established. 


To this point, the chapter described six steps involved in designing performance 
assessments in relation to a set of professional standards. The process includes identifying 
key performances, developing activities that elicit the performances, and writing 
assessment rubrics to guide judgments about the quality of the performances. The next 
section contains a brief overview of the remaining three phases (6 steps) of high quality 
assessment outlined by Dr. Anderson viewed from the perspective of performance 
assessment design. 


Evaluation, Implementation, and Program Development Phase 


When the design phase is complete, the quality assessment process enters the evaluation 
phase, which consists of two steps; piloting the performance and using the rubric to 
evaluate samples of student work. 


The goal of piloting is to seek feedback from students and faculty who may not have been 
directly involved in the design phase prior to implementation of the assessment at the 
program or department level. What appeared to be crystal clear during the writing process 


often turns out to be confusing or cumbersome when one actually tries to follow the 
directions or use the rubric to assess students’ work. Piloting allows tweaking to occur 
before final implementation. Once a performance assessment is rolled out for use it is 
good practice to establish a review process that ensures monitoring and adjustment occurs 
over time. 


During piloting actual student work is evaluated for the first time using the rubric that was 
developed and a content review of rated material presents an opportunity to refine the 
evaluation assessment standards that were established at the conclusion of the design 
phase. The process of selecting anchor performances, examples of student work at each 
level of the rubric, provides insight regarding how the assessment activity, assignment 
template and rubric are interacting to elicit the performance and guide documentation. 
Content analysis of anchor documents that meet or exceed standards is one way to ensure 
that the assessment is valid. 


The evaluation phase ensures that the components of the assessment design are coherent, 
understandable, and differentiate performances as expected. Evaluation provides 
opportunity for quality control before going to scale. During the implementation phase 
that follows, faculty assessors are trained using the anchor performances to increase inter- 
rater reliability and the performance assessment becomes a program requirement for all 
students. 


Program development is the final phase in the high quality assessment process. Among 
other things, this involves a close examination of achievement results to see if there might 
be evidence of unintended bias related to how the assessment was designed and written. 
Analysis of student outcomes and interpretation of the patterns of performance that 
emerge engage faculty in dialogue about the interrelationships among curriculum, 
instruction and assessment. These conversations inform the re-design phase as the next 
cycle of high quality assessment begins. 


Conclusion 


This chapter described high quality assessment practices associated with the design of 
performance assessments based on standards for Educational Leadership. Based on our 
experience at EMU, the process is involved and time-consuming but we have found that 
the payoff regarding program development is worth it in the end. 


Some professors of educational leadership will engage in this work because they value 
performance assessment and believe it is the right thing to do. Others will do it because it 
is mandated by accreditation agencies or institutional policy. Regardless of the motivation 
for one’s engagement, we firmly believe the work should be of high quality. This 
overview of one strategy for designing performance assessment is an attempt to point us 
in that direction. 
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Appendix 


Note:Excerpt from: Performance expectations and indicators for education leaders: An 
ISLLC-based guide to implementing leader standards and a common guide to the 
educational leadership policy standards. 


Performance Expectation 2: Teaching and Learning 


Education leaders ensure achievement and success of all students by monitoring and 
continuously improving teaching and learning. 


Dispositions exemplified in Expectation 2: 
Education leaders believe in, value, and are committed to 


Learning as the fundamental purpose of school 
Diversity as an asset 

Continuous professional growth and development 
Livelong learning 

Collaboration with all stakeholders 

High expectations for all 

Student learning 


Narrative 


A strong, positive, professional culture fosters learning by all educators and students. In a 
strong professional culture, leaders share and distribute responsibilities to provide quality, 
effectiveness, and coherence across all components of the instructional system (such as 
curriculum, instructional materials, pedagogy, and student assessment). Leaders are 
responsible for a professional culture in which learning opportunities are targeted to the 
vision and goals and differentiated appropriately to meet the needs of every student. 
Leaders need knowledge, skills, and beliefs that provide equitable differentiation of 
instruction and curriculum materials to be effective with a range of student characteristics, 
needs, and achievement. 


A strong professional culture includes reflection, timely and specific feedback that 
improves practice, and support for continuous improvement toward vision and goals for 
student learning. Educators plan their own professional learning strategically, building 
their own capacities on the job. Leaders engage in continuous inquiry about effectiveness 
of curricular and instructional practices and work collaboratively to make appropriate 
changes that improve results. 


Element A: Strong Professional Culture 


A strong professional culture supports teacher learning and shared commitments to the 
vision and goals. 


Indicators: A leader... 


1. Develop shared understanding, capacities, and commitment to high expectations for 
all students and closing achievement gaps. 

2. Guides and supports job-embedded, standards-based professional development that 
improves teaching and learning and meets diverse learning needs of every student. 

3. Models openness to change and collaboration that improves practices and student 
outcomes. 

4. Develops time and resources to build a professional culture of openness and 
collaboration, engaging teachers in sharing information, analyzing outcomes, and 
planning improvement. 

5. Provides support, time, and resources for leaders and staff to examine their own 
beliefs, values, and practices in relation to the vision and goals for teaching and 
learning. 

6. Provides ongoing feedback using data, assessments, and evaluation methods that 
improve practice. 

7. Guides and monitors individual professional development plans and progress for 
continuous improvement of teaching and learning. 


Element B: Rigorous Curriculum and Instruction 


Improving achievement of all students requires all educators to know and use rigorous 
curriculum and effective instructional practices, individualized for success of every 


student. 


Indicators: A leader... 


i 


Develops shared understanding of rigorous curriculum and standards-based 
instructional programs, working with teams to analyze student work, monitor student 
progress, and redesign curricular and instructional programs to meet diverse needs. 


. Provides coherent, effective guidance of rigorous curriculum and instruction, 


aligning content standards, curriculum, teaching, assessments, professional 
development, assessments, and evaluation methods. 


. Provides and monitors effects of differentiated teaching strategies, curricular 


materials, educational technologies, and other resources appropriate to address 
diverse student populations, including students with disabilities, cultural and 
linguistic differences, gifted and talented, disadvantaged social economic 
backgrounds, or other factors affecting learning. 


. Identifies and uses high-quality research and data-based strategies and practices that 


are appropriate in the local context to increase learning for every student. 


Element C: Assessment and Accountability 


Improving achievement and closing achievement gaps require that leaders make 
appropriate, sound use of assessments, performance management, and accountability 
strategies to achieve vision, mission, and goals. 


Indicators: A leader... 


1 


2: 


. Develops and appropriately uses aligned, standards-based accountability data to 


improve the quality of teaching and learning. 

Uses varied sources and kinds of information and assessments (such as test scores, 
work samples, and teacher judgment) to evaluate student learning, effective teaching, 
and program quality. 


. Guides regular analyses and disaggregation of data about all students to improve 


instructional programs. 


. Uses effective data-based technologies and performance management systems to 


monitor and analyze assessment results for accountability reporting and to guide 
continuous improvement. 


. Interprets data and communicates progress toward vision, mission, and goals for 


educations, the school community, and other stakeholders. 


Source: Sanders, N. M., & Kearney, K. M. (Eds.). (2008). Performance expectations and 
indicators for education leaders: An ISLLC-based guide to implementing leader 
standards and a companion guide to the educational leadership policy standards: ISLLC 
2008. Washington, DC: State Consortium on Education Leadership, Council of Chief 
State School Officers (pp. 16-18). 


